Surgical simulation model generating method, surgical simulation method, and surgical simulator

ABSTRACT

A surgical simulation model generating method which includes a first process in which a computing unit acquires geometrical information of an organ from a medical image stored in a storage unit, including an image of the organ, and generates volume data for the organ; a second process in which, after the first process, the computing unit forms nodal points by meshing the organ represented by the generated volume data; a third process in which the computing unit generates a simulated membrane that covers the organ represented by the volume data meshed in the second process; and a fourth process in which the computing unit generates a simulated organ by drawing an imaginary line so as to extend from each nodal point formed on a surface of the organ represented by the volume data meshed in the second process in a direction that intersects the simulated membrane.

TECHNICAL FIELD

The present invention relates to a method for generating a surgicalsimulation model used when conducting a surgical simulation beforeperforming a surgical operation using an endoscope, and also relates toa surgical simulation method and a surgical simulator.

BACKGROUND ART

With advances in medical technology and medical instruments, manyabdominal surgical operations are being performed using a laparoscope.Since laparoscopic surgery is performed by viewing a three-dimensionalobject displayed on a two-dimensional image display device, training isindispensable for acquiring of the required skill. In actuallaparoscopic surgery, the surgery must be planned so as to match eachindividual patient because the number of blood vessels, the positions ofthe blood vessels, and the positional relationship of organs, forexample, the position and size of a tumor, differ from patient topatient.

For this purpose, it may be appropriate to perform, prior to surgery, asurgical simulation based on information acquired of each individualpatient.

To acquire information of each individual patient, it is common to usemedical image data such as CT or MRI data, but images of the membranetissues surrounding the organ to be operated on cannot be captured bysuch means. Because of the inability to recognize such membrane tissues,there arises the problem that the membrane tissues cannot be modeled. Amodel that does not incorporate membrane tissues is unsuitable for usein a preoperative simulation. On the other hand, to compute the motionof an organ model at high speed, the physical and dynamic conditions ofthe model of the organ to be operated on may be set linearly. However,in this case, the deformation of the organ model would greatly differfrom the actual deformation, rendering such a model unsatisfactory foruse in a preoperative simulation.

Further, such a surgical simulator is equipped with a force sensingdevice that produces the reaction of a simulated organ that matches theposition of the simulated surgical instrument being manipulated by thesurgical simulation operator and the position where it touches thesimulated organ. However, it is not common to compute the reaction ofthe simulated organ and supply the computed reaction to the forcesensing device, while at the same time, computing in real time theposition achieved by the motion of the simulated organ.

Further, in a prior art surgical simulation model of a simulated organthat uses a finite-element method, volume data for an organ, forexample, is meshed to generate a simulated organ segmented into aplurality of tetrahedrons. Then, a stiffness matrix that describes thedynamic property of the simulated organ is generated by applying Young'smodulus or Poisson's ratio or the like as physical values to thetetrahedrons. Then, the motion equation of the simulated organ that usesthe stiffness matrix is solved by numerical computation, therebysimulating the motion of the simulated organ.

However, since it takes a finite time to complete the numericalcomputation of the motion equation that uses such a stiffness matrix, ithas not been possible Lo compute the motion of the simulated organ inreal time. Furthermore, the computation using the prior art stiffnessmatrix has had the problem that the computation may diverge.

PRIOR ART DOCUMENTS Patent Documents

Patent Document 1: Japanese Patent No. 4155637

Patent Document 2: Japanese Patent No. 4117949

Patent Document 3: Japanese Patent No. 4117954

Patent Document 3: Japanese Patent No. 4290312

SUMMARY OF THE INVENTION Problem to be Solved by the Invention

The problem to be solved by the invention is to provide a simulationmodel generating method, a surgical simulation method, and a surgicalsimulator that can compute the motion of a simulated organ in real time.Means for Solving the Problem

The surgical simulation model generating method according to the presentinvention includes: a first process in which a computing unit acquiresgeometrical information of an organ from a medical image stored in astorage unit, including an image of the organ, and generates volume datafor the organ; a second process in which, after the first process, thecomputing unit forms nodal points by meshing the organ represented bythe generated volume data; a third process in which the computing unitgenerates a simulated membrane that covers the organ represented by thevolume data meshed in the second process; and a fourth process in whichthe computing unit generates a simulated organ by drawing an imaginaryline so as to extend from each nodal point formed on a surface of theorgan represented by the volume data meshed in the second process in adirection that intersects the simulated membrane and thereby forming amembrane nodal point at a point where the imaginary line intersects thesimulated membrane generated in the third process, and by arranging oneach imaginary line an imaginary inter-membrane spring that connectsbetween the nodal point formed on the surface of the organ and themembrane nodal point, while also arranging an in-plane spring thatconnects between adjacent membrane nodal points on the simulatedmembrane.

Another surgical simulation model generating method according to thepresent invention includes: a first process in which a computing unitacquires geometrical information of an organ from a medical image storedin a storage unit, including an image of the organ, and generates volumedata for the organ; a second process in which, after the first process,the computing unit forms nodal points by meshing the organ representedby the generated volume data; and a third process in which the computingunit generates a simulated organ by arranging an imaginary spring so asto connect between each of the nodal points on the organ represented bythe meshed volume data.

A surgical simulation method according to the present inventionincludes: a force sensing simulation process in which a computing unitcauses a force sensing device to produce reaction of a simulated organthat matches the position of a simulated surgical instrument beingmanipulated by a surgical simulation operator and the position where thesimulated surgical instrument touches the simulated organ; a simulatedmotion computing process in which the computing unit acquires, from astorage unit, surgical simulation model data for a simulated organhaving an organ represented by meshed volume data and a simulatedmembrane covering the organ represented by the meshed volume data, thesimulated organ being generated by drawing an imaginary line so as toextend from each nodal point formed on a surface of the organrepresented by the meshed volume data in a direction that intersects thesimulated membrane and thereby forming a membrane nodal point at a pointwhere the imaginary line intersects the simulated membrane, and byarranging on each imaginary line an imaginary inter-membrane spring thatconnects between the nodal point formed on the surface of the organ andthe membrane nodal point, while also arranging an in-plane spring thatconnects between adjacent membrane nodal points on the simulatedmembrane, and the computing unit then computes the reaction of thesimulated organ due to a movement of the simulated surgical instrumentand the touching of the simulated organ with the simulated surgicalinstrument in the force sensing simulation process, and supplies thecomputed reaction to the force sensing simulation process, while at thesame time, computing the position achieved by the motion of thesimulated organ; an image generation process in which the computing unitgenerates, based on the position of the simulated organ computed in thesimulated motion computing process, a simulated image of the simulatedorgan as seen from a simulated endoscope; and an image display processin which the computing unit displays the image generated in the imagegeneration process on a display unit.

Another surgical simulation method according to the present inventionincludes: a force sensing simulation process in which a computing unitcauses a force sensing device to produce reaction of a simulated organthat matches the position of a simulated surgical instrument beingmanipulated by a surgical simulation operator and the position where thesimulated surgical instrument touches the simulated organ; a simulatedmotion computing process in which the computing unit acquires, from astorage unit, surgical simulation model data for a simulated organgenerated by arranging an imaginary spring so as to connect between eachnodal point formed by meshing an organ represented by volume data,computes the reaction of the simulated organ due to a movement of thesimulated surgical instrument and the touching of the simulated organwith the simulated surgical instrument in the force sensing simulationprocess, and supplies the computed reaction to the force sensingsimulation process, while at the same time, computing the positionachieved by the motion of the simulated organ; an image generationprocess in which the computing unit generates, based on the position ofthe simulated organ computed in the simulated motion computing process,a simulated image of the simulated organ as seen from a simulatedendoscope; and an image display process in which the computing unitdisplays the image generated in the image generation process on adisplay unit.

A surgical simulator according to the present invention includes: asurgical simulation model data unit which stores surgical simulationmodel data for a simulated organ having an organ represented by meshedvolume data and a simulated membrane covering the organ represented bythe meshed volume data, the surgical simulation model data beinggenerated by drawing an imaginary line so as to extend from each nodalpoint formed on a surface of the organ represented by the meshed volumedata in a direction that intersects the simulated membrane and therebyforming a membrane nodal point at a point where the imaginary lineintersects the simulated membrane, and by arranging on each imaginaryline an imaginary inter-membrane spring that connects between the nodalpoint formed on the surface of the organ and the membrane nodal point,while also arranging an in-plane spring that connects between adjacentmembrane nodal points on the simulated membrane; a force sensing devicewhich produces the reaction of the simulated organ that matches theposition of a simulated surgical instrument being manipulated by asurgical simulation operator and the position where the simulatedsurgical instrument touches the simulated organ; a simulated motioncomputing unit which acquires the surgical simulation model data fromthe surgical simulation model data unit, computes the reaction of thesimulated organ due to a movement of the simulated surgical instrumentand the touching of the simulated organ with the simulated surgicalinstrument at the force sensing device, and supplies the computedreaction to the force sensing device, while at the same time, computingthe position achieved by the motion of the simulated organ; an imagegenerating unit which generates, based on the position of the simulatedorgan computed by the simulated motion computing unit, a simulated imageof the simulated organ as seen from a simulated endoscope; and an imagedisplay unit which displays the image generated by the image generatingunit.

Another surgical simulator according to the present invention includes:a Surgical simulation model data unit which stores surgical simulationmodel data for a simulated organ generated by arranging an imaginaryspring so as to connect between each nodal point formed by meshing anorgan represented by volume data; a force sensing device which producesthe reaction of the simulated organ that matches the position of asimulated surgical instrument being manipulated by a surgical simulationoperator and the position where the simulated surgical instrumenttouches the simulated organ; a simulated motion computing unit whichacquires the surgical simulation model data from the surgical simulationmodel data unit, computes the reaction of the simulated organ due to amovement of the simulated surgical instrument and the touching of thesimulated organ with the simulated surgical instrument at the forcesensing device, and supplies the computed reaction to the force sensingdevice, while at the same time, computing the position achieved by themotion of the simulated organ; an image generating unit which generates,based on the position of the simulated organ computed by the simulatedmotion computing unit, a simulated image of the simulated organ as seenfrom a simulated endoscope; and an image display unit which displays theimage generated by the image generating unit.

Advantageous Effect of the Invention

According to the simulation model generating method, surgical simulationmethod, and surgical simulator of the invention described above, theposition achieved by the motion of the simulated organ can be computedin real time.

The object and advantages of the invention will be realized and attainedby means of the elements and combinations particularly pointed out inthe claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and arenot restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow diagram for explaining a first embodiment of a surgicalsimulation model generating method.

FIG. 2 is a flow diagram for explaining a second embodiment of asurgical simulation model generating method.

FIG. 3 is a flow diagram for explaining a first embodiment of a surgicalsimulation method.

FIG. 4 is a flow diagram for explaining a second embodiment of asurgical simulation method.

FIG. 5A is a functional block diagram for explaining an embodiment of asurgical simulator.

FIG. 5B is a functional block diagram of a computer.

FIG. 6 is a diagram for explaining the structure of a membrane.

FIG. 7 is a diagram for explaining a finite-element model of an organ.

FIG. 8 is a diagram for explaining the characteristic of an imaginaryspring.

FIG. 9 is a diagram showing the relationship between <K_(f)>, <K(<U>)>,and <U>.

MODES FOR CARRYING OUT THE INVENTION

FIG. 1 is a flow diagram for explaining a first embodiment of a surgicalsimulation model generating method. FIG. 2 is a flow diagram forexplaining a second embodiment of a surgical simulation model generatingmethod. FIG. 3 is a flow diagram for explaining a first embodiment of asurgical simulation method. FIG. 4 is a flow diagram for explaining asecond embodiment of a surgical simulation method. FIG. 5A is afunctional block diagram for explaining an embodiment of a surgicalsimulator. FIG. 5B is a functional block diagram of a computer.

In FIG. 5A, reference numeral 501 is a medical image data storage unit,502 is a volume data constructing unit, 503 is an image generating unit,504 is an image display device, 505 is a surgical simulation model dataunit, 506 is a force sensing device, 507 is a simulated motion computingunit, 203 is a simulated surgical instrument, 509 is a simulatedendoscope, and 510 is a simulated forceps. The simulated surgicalinstrument 203 includes the simulated surgical instrument 203 and thesimulated endoscope 509.

FIG. 5B is a functional block diagram of a computer which implementssome of the functions shown in FIG. 5A. The computer includes acomputing unit 520, a storage unit 521, a display unit 522, an inputunit 523, and a communication unit 534. The surgical simulation modelgenerating method and the surgical simulation method are each realizedby the computer executing a prescribed program. Further, the functionsof the surgical simulator shown in FIG. 5A, except the functions of theforce sensing device 506 and the simulated surgical instrument 203, areimplemented by the computer executing a prescribed program. Here, thecomputing unit 520 controls the force sensing device 506 and thesimulated surgical instrument 203 by performing communications with themusing the communication unit 534. The image display device 504 isrealized by the display unit 522.

Embodiment 1

The first embodiment of the surgical simulation model generating methodwill be described below. The surgical simulation model generationaccording to the first embodiment is performed using the surgicalsimulator shown in FIG. 5A.

The medical image data storage unit 501 stores the source data ofmedical images including, for example, those of the organ to be operatedon. The source data of medical images is obtained, for example, by CTimaging or MRI imaging.

The image generating unit 503 generates images, including those oforgans, by using the medical image data stored in the medical image datastorage unit 501. The images, each representing a cross section of thepatient to be operated on, are obtained by scanning the patient's bodyin thin slices in a prescribed direction. The volume data constructingunit 502 acquires geometrical information of each organ from the medicalimage data obtained by capturing the images of body parts, includingthose of the organ, while viewing the medical images, including those ofthe organ, displayed on the image display device 504. Then, based on thegeometrical information of each organ thus acquired, the volume dataconstructing unit 502 extracts each body part (organ) two-dimensionallyfrom the medical image data, and generates images by arranging thetwo-dimensionally extracted body parts in accordance with theirpredetermined positions relative to each other. Further, the volume dataconstructing unit 502 constructs three-dimensional volume data of eachbody part (P101 in FIG. 1) by stacking one on top of another thetwo-dimensional images of each body part extracted from the images ofthe thin slices. The three-dimensional volume data is stored in themedical image data storage unit 501 in a storage area different from thestorage area where the source data of the medical images is stored. Thevolume data of each organ can be generated using, for example, a priorknown method.

The operator of the surgical simulator causes the image generating unit503 to retrieve from the medical image data storage unit 501 the medicalimage data of the patient to be operated on. The image generating unit503 extracts the organs situated in the predetermined area containingthe intended organ for which the data has been retrieved, and displaysthem on the image display device 504 in accordance with theirpredetermined positions relative to each other.

The operator reorients the intended organ by moving and/or rotating thevolume data of the intended organ, as needed, by selecting it from amongthe organs displayed on the image display device 504. This is done sothat the orientation of the organ based on the volume data matches theorientation of the organ as seen from an endoscope in actual surgery.

The volume data has an ID assigned to each organ. The ID of the originalvolume data before the move/rotate is designated as ID=F(x,y,z), and thevolume data after he move/rotate as ID=G(x,r,z). When this move/rotatetransformation is expressed in the form of a matrix R_(ID), therelationship between F and G is given as R_(ID)F=G, so that G can beobtained as G−R_(ID) ⁻¹F. The moved position data, including the matrixR_(ID), is stored in the surgical simulation model data unit 505.

Next, as the intended organoved as described above, the volume data andthe moved position data are also generated for other organs connected tothe intended organ. The model data, etc., for these other organs arealso stored in the surgical simulation model data unit 505.

Then, for each organ represented by the volume data, the volume dataconstructing unit 502 takes the geometrical information such as meshspacing as input information, and generates, using a program, afinite-element model of the organ by meshing the volume data withtetrahedrons and thus forming nodal points thereon based on itsanatomical properties (P102 in FIG. 1).

Next, for the organ represented by the volume data, the volume dataconstructing unit 502 takes the geometrical information such as meshspacing as input information, and generates, based on the anatomicalproperties, a plurality of simulated membranes around the organrepresented by the designated volume data (P103 in FIG. 1). The spacingfrom one simulated membrane to another is determined based on thethicknesses of the simulated membranes generated. The spacing betweenthe organ represented by the volume data and the simulated membranescovering the organ may be constant or may be varied partially. Further,the spacing between the plurality of simulated membranes covering theorgan represented by the volume data may be made the same or may be madedifferent between the respective membranes (P103 a and P103 b in FIG.1). In this way, a model that can variously change the deformation ofthe simulated membranes can be generated.

More specifically, as shown in FIG. 6, the volume data constructing unit502 draws imaginary lines 603 a, 603 b, 603 c, . . . so as to extendfrom the nodal points 602 a, 602 b, 602 c, . . . formed on the surfaceof the organ 601 represented by the volume data meshed in process P102,in directions that intersect the simulated membranes 604 a, 604 b, 604c, . . . Each imaginary line extends in a direction normal to thesimulated membranes.

Then, the volume data constructing unit 502 forms membrane nodal points605 a, 605 b, 605 c, . . . at the points where the imaginary lines 603a, 603 b, 603 c, . . . intersect the simulated membranes 604 a, 604 b,604 c, . . . generated in process P103. Each imaginary line intersectsthe plurality of simulated membranes, and the membrane nodal points areformed one at each intersection.

Then, the volume data constructing unit 502 arranges, on each of theimaginary lines 603 a, 603 b, 603 c, . . . , imaginary inter-membranesprings 606 a, 606 b, 606 c, . . . that connect between a correspondingone of the nodal points 602 a, 602 b, 602 c, . . . formed on the surfaceof the organ and a corresponding one of the membrane nodal points 605 a,605 b, 605 c, . . . and between the corresponding membrane nodal pointson any two adjacent simulated membranes (P104 in FIG. 1).

Further, the volume data constructing unit 502 arranges in-plane springs607 a, 607 b, 607 c, . . . each connecting between adjacent membranenodal points on a corresponding one of the simulated membranes (P104 inFIG. 1). In this way, a simulated organ having the organ represented bythe meshed volume data and the simulated membranes covering the organrepresented by the meshed volume data is generated.

The imaginary inter-membrane springs 606 a, 606 b, 606 c, . . . and thein-plane springs 607 a, 607 b, 607 c, . . . are each formed using aspring model.

More specifically, the imaginary inter-membrane springs 606 a, 606 b,606 c, . . . arranged between the respective nodal points formed on thesurface of the organ may be chosen to have different spring constants k(k=k1, k2, . . . ), and the in-plane springs 607 a, 607 b, 607 c, . . .may also be chosen to have different spring constants K (K=K1, K2, . . .) (P104 a and P104 b in FIG. 1). Alternatively, all of the springconstants k or all of the spring constants K may be made the same. Inthis way, when force is applied to the simulated membranes, thecorresponding portions of the simulated membranes do not deformuniformly along the imaginary line direction but deform differentlyaccording to the different spring constants. Further, when force isapplied to the simulated membranes, each simulated membrane does notdeform uniformly in the plane but deforms differently according to thedifferent spring constants. The values of the spring constants k and Kcan be set based, for example, on anatomical data.

Furthermore, the imaginary inter-membrane springs 606 a, 606 b, 606 c, .. . and the in-plane springs 607 a, 607 b, 607 c, . . . may each has aphysical constant such that the spring breaks when a predeterminedtensile force or stretching force is applied (P104 c in FIG. 1). Thissimulates the simulated membrane being torn off when a force is appliedto the simulated membrane. This physical constant can be set based, forexample, on anatomical data.

Then, the volume data constructing unit 502 stores the surgicalsimulation model data for such a simulated organ into the surgicalsimulation model data unit 505.

According to the surgical simulation model generating method of thefirst embodiment described above, since the simulation accuracy ofmembrane deformation can be improved with the spring forces acting inthe plane of each simulated membrane, a model is generated that makes itpossible to achieve a surgical simulation having a high training effect.

Embodiment 2

Next, a method for generating a surgical simulation model intended tosimulate a large deformation of a designated organ with high accuracywill be described below with reference to FIG. 2. The surgicalsimulation model generation according to the second embodiment also isperformed using the surgical simulator shown in FIG. 5A.

In FIG. 2, processes P201 and P202 are the same as the correspondingprocesses P101 and P102 shown in FIG. 1.

Next, in process P203, the volume data constructing unit 502 generates asimulated organ by arranging imaginary springs 701 a, 701 b, 701 c, . .. so as to interconnect the respective nodal points on thefinite-element model of the organ represented by the volume data meshedwith tetrahedrons (FIG. 7). Each imaginary spring is arranged betweenadjacent nodal points on the organ represented by the volume data.

The imaginary springs 701 a, 701 b, 701 c, . . . are each formed using aspring model. The imaginary springs are arranged between the respectivenodal points on the surface of the organ and between the respectivenodal points inside the organ. As in the first embodiment, the imaginarysprings may include those having different spring constants. The springconstants of the imaginary springs can be set based, for example, onanatomical data.

Further, the imaginary springs 701 a, 701 b, 701 c, . . . may each has aphysical constant such that the spring breaks when a prescribed tensileforce or stretching force is applied. In this case, the imaginarysprings 701 a, 701 b, 701 c, . . . may each has a characteristic that isnot linear with respect to the applied tensile force or stretchingforce. For example, each imaginary spring may have a nonlinearcharacteristic that is convex upward as shown in FIG. 8, and may beconstructed to break when the prescribed tensile force or stretchingforce is applied. The nonlinear physical characteristics of theimaginary springs can be set based, for example, on anatomical data.

In this way, the simulated organ is generated by arranging the imaginarysprings so as to interconnect the respective nodal points formed bymeshing the organ represented by the volume data.

Then, the volume data constructing unit 502 stores the surgicalsimulation model data for such a simulated organ into the surgicalsimulation model data unit 505.

According to the surgical simulation model generating method of thesecond embodiment described above, since the simulation accuracy ofsimulated organ deformation can be improved with the spring forcesacting on the simulated organ, a model is generated that makes itpossible to achieve a surgical simulation having a high training effect.

Embodiment 3

Next, a description will be given of a surgical simulation that uses thesurgical simulation model data generated according to the firstembodiment and an apparatus that is used to perform the simulation. Thesurgical simulation according to the third embodiment is performed usingthe surgical simulator shown in FIG. 5A

The image generating unit 503 retrieves from the surgical simulationmodel data unit 505 the surgical simulation model data for the simulatedorgan having the organ represented by the meshed volume data and thesimulated membranes covering the organ represented by the meshed volumedata. Then, the image generating unit 503 causes the image displaydevice 504 to display the simulated organ having the simulated organ andthe plurality of simulated membranes arranged around the organ. In thissimulated organ, the imaginary inter-membrane springs and the in-planesprings are arranged as earlier described, defining the dynamicproperties of the simulated organ.

The surgical simulation operator of the surgical simulator touches asimulated membrane forming part of the simulated organ by manipulatingthe simulated forceps 510 as one of the simulated surgical instruments203, and pulls the simulated membrane by holding it between the pincersof the simulated forceps 510. The reaction of the simulated organ thatmatches the position of the simulated forceps 510 and the position whereit touches the simulated membrane is produced by the force sensingdevice 506, and this reaction is fed back to the simulated forceps 510through the force sensing device 506 (P301). As the force sensing device506, use may be made, for example, of a prior known force sensingdevice.

To produce the reaction of the simulated organ, the simulated motioncomputing unit 507 acquires the surgical simulation model data for thesimulated organ from the surgical simulation model data unit 505, andcomputes the reaction of the simulated organ due to the movement of thesimulated forceps 510 and the touching of the simulated organ with thesimulated forceps 510 at the force sensing device 506 (P302). Further,the simulated motion computing unit 507 supplies the thus computedreaction to the force sensing device 506, while at the same time,computing the position achieved by the motion of the simulated organ(P302).

More specifically, the reaction to be applied to the simulated forceps510 is computed by the simulated motion computing unit 507, based on thespring constants of the imaginary inter-membrane springs 606 a, 606 b,606 c, . . . and in-plane springs 607 a, 607 b, 607 c, . . . and on themoved position of the simulated forceps 510. The simulated motioncomputing unit 507 causes the force sensing device 506 to produce thecomputed reaction as a tensile force, and the reaction is thussimulated. The gripping force exerted by the simulated forceps differsdepending on the position of the pincers of the simulated forceps 510,and this gripping force is computed by the simulated motion computingunit 507. When the stretching or tensile force with which the simulatedmembrane is being pulled by the simulated forceps 510 reaches or exceedsa predetermined value, the imaginary inter-membrane springs 606 a, 606b, 606 c, . . . and the in-plane springs 607 a, 607 b, 607 c, . . . arecaused to break, thus simulating the simulated membrane being torn off(P302). In this way, when the simulated forceps is pulled with apredetermined tensile force or stretching force, the imaginaryinter-membrane springs or the in-plane springs are caused to break, thusapplying the sensation of tearing off the simulated membrane to thesimulated forceps. The surgical simulator can thus simulate thesimulated membrane being torn off when a force is applied to it.

Preferably, the simulated motion computing unit 507 performs the abovecomputations in real time in the surgical simulation. The real timecomputing method of the simulated motion computing unit 507 is the sameas the method to be described later as a fourth embodiment, and willtherefore be described in detail later. Further, to enhance thecomputational speed, the simulated motion computing unit 507 may beimplemented using a computer different from the computer used toimplement the other functions. Performing the surgical simulation inreal time means performing the surgical simulation at about the samespeed as the actual surgery will be performed.

During the surgical simulation, the image generating unit 503 generatessimulated images of the simulated organ and simulated membranes as seenfrom the simulated endoscope, based on the positions of the simulatedorgan and simulated membranes computed by the simulated motion computingunit 507. In this way, the image generating unit 503 generatesdynamically simulated images of the simulated organ and the simulatedsurgical instructions 203 including the simulated forceps, as if theimages were being actually captured by the simulated endoscope (P303).As the image generating unit 503, use may be made, for example, of aprior known image generating unit.

The simulated images are displayed on the image display device 504, andthe surgical simulation operator performs the surgical simulation whileviewing the displayed images (P304).

According to the surgical simulation method and surgical simulator ofthe third embodiment described above, the motion of each simulatedmembrane can be computed in real time. Further, since the simulationaccuracy of membrane deformation can be improved with the spring forcesacting in the plane of each simulated membrane, a surgical simulationhaving a high training effect can be achieved. Furthermore, since thesimulation accuracy of simulated organ deformation can be improved inthe simulation of a large deformation of the simulated organ, a surgicalsimulation having a high training effect can be achieved. Moreover,since the tensile force exerted when the simulated membrane is pulled isfed back to the simulated forceps through the force sensing device, asimulation with enhanced reality can be achieved.

Embodiment 4

Next, a description will be given of a surgical simulation that uses thesurgical simulation model data generated according to the secondembodiment and an apparatus that is used to perform the simulation. Thesurgical simulation according to the fourth embodiment is performedusing the surgical simulator shown in FIG. 5A.

The image generating unit 503 acquires from the surgical simulationmodel data unit 505 the surgical simulation model data for the simulatedorgan generated by arranging the imaginary springs so as to interconnectthe respective nodal points formed by meshing the organ represented bythe volume data. Then, the image generating unit 503 generates images ofthe simulated organ including the simulated organ, and displays them onthe image display device 504.

The surgical simulation operator of the surgical simulator touches thesimulated organ forming part of the simulated organ by manipulating thesimulated forceps 510 as one of the simulated surgical instruments 508,and pushes or pulls the simulated organ by holding it between thepincers of the simulated forceps 510. The reaction that matches theposition of the simulated forceps 510 and the position where it touchesthe simulated organ is produced by the force sensing device 506 (P401).

To produce the reaction of the simulated organ, the simulated motioncomputing unit 507 acquires the surgical simulation model data for thesimulated organ from the surgical simulation model data unit 505, andcomputes the reaction of the simulated organ due to the movement of thesimulated surgical instrument and the touching of the simulated organwith the simulated surgical instrument at the force sensing device 506(P402). Further, the simulated motion computing unit 507 supplies thethus computed reaction to the force sensing device 506, while at thesame time, computing the position achieved by the motion of thesimulated organ (P402).

More specifically, the reaction to be applied to the simulated forceps510 is computed by the simulated motion computing unit 507, based on thenonlinear spring constants of the imaginary springs 701 a, 701 b, 701 c,. . . and on the moved position of the simulated forceps. The simulatedmotion computing unit 507 causes the force sensing device 506 to producethe computed reaction as a compression or tensile force, and thereaction is thus simulated. The gripping force exerted by the simulatedforceps differs depending on the position of the pincers of thesimulated forceps, and this gripping force is computed by the simulatedmotion computing unit 507.

A nonlinear FEM (Finite-Element Method) is used to generate an organdeformation model in order to simulate the deformation of the simulatedorgan with high accuracy.

The nonlinear process is performed in real time by piecewise linearizingthe time evolution of the nonlinear process. In the prior art linearcomputation model, denoting the displacement vector as <U>, the massmatrix as <M>, the viscosity resistance matrix as <C>, the stiffnessmatrix <K>, and the external force <f>, and assuming that thedisplacement is small, the motion equation is defined by the followingequation (1). (In this specification, the notation <a> denotes a vectoror matrix of “a”, and is shown in boldface.)

MÜ+C{dot over (U)}+K(U)=f  (1)

In the fourth embodiment, the stiffness matrix K in equation (1)describes the physical values of the simulated organ and is generatedusing the spring constants of the imaginary springs. In the prior artsimulated organ, the stiffness matrix was generated by using, forexample, the physical values applied to the tetrahedrons of thefinite-element mode. Further, in the fourth embodiment, the stiffnessmatrix <K> is given as the following nonlinear model <K(<U>)> which is afunction of the displacement matrix <U>.

$\begin{matrix}{{{{M\overset{¨}{U}} + {C\overset{.}{U}} + {K_{f}(U)}} = f}{{K_{f}(U)} = {\int_{0}^{U}{{K(u)}\ {u}}}}} & (2)\end{matrix}$

When the stiffness matrix <K> is given as <K(<U>)>, i.e., as a functionof the displacement matrix <U>, the reaction of the simulated organ canbe computed based on the physical values of the simulated organ obtainedby varying the physical values according to the positional displacementof the simulated organ caused by the force applied to the simulatedorgan. In this way, since the simulation accuracy of simulated organdeformation can be improved in the simulation of a large deformation ofthe simulated organ, a surgical simulation having a high training effectcan be achieved.

(Piecewise Linearization)

IF force is applied to an object (for example, the simulated organ) byusing a simulated surgical instrument such as a simulated forceps, theobject is deformed, generating stress; then, the deformation of theobject ceases when the surface force of the object equilibrates with theforce exerted by the simulated surgical instrument, and the equilibriumof the forces is thus reached. When the force is further applied, theobject is further deformed until the surface force is generated toequilibrate with it and, when the equilibrium of the forces is reached,the deformation is stabilized. In real time processing, this process isrepeated at high speed.

When such dynamic computations are assumed, it is considered that thedifference between the stiffness matrix one frame back in time(stress-generating source information) and the current stiffness matrixis small (i.e., piecewise linearized). In particular, in the case of thesurgical simulator, the operation is relatively mild. Hence, using thestiffness matrix <K_(f)>_(i) corresponding to the position (<U>^(k-1))one frame back in time and the amount of displacement, Δ<u>^(k) _(i),during one frame, the following equation (3) is obtained.

K _(fi)(u _(i) ^(k))=K _(fi)(u _(i) ^(k-1))+K _(fi)(u _(i) ^(k-1))·Δu_(i) ^(k-1)  (3)

Accordingly, the process proceeds as follows:

(1) K _(fi)(u _(i) ^(k))=K _(fi)(u _(i) ^(k-1))+K _(fi)(u _(i)^(k-1))·Δu _(i) ^(k-1)

(2) α_(i) ^(k)=(f _(i) ^(k) −C _(i) ·v _(i) ^(k-1) −K _(fi)(u _(i)^(k)))/M _(i)

(3) v _(i) ^(k) =v _(i) ^(k-1)+α_(i) ^(k) Δt

(4) Δu_(i) ^(k-1)=v_(i) ^(k)Δt

(5) u _(i) ^(k) =u _(i) ^(k-1) +Δu _(i) ^(k)

(6) K_(fi) ^(k) is calculated using u_(i) ^(k).

(7) The process returns to (1) (until i=N)  (4)

The entire process from (1) to (7) forms equation (4). The superscript krepresents the computation time instant, α_(i) represents theacceleration of the i-th element, f_(i) represents the external force ofthe same, v_(i) represents the velocity of the same, u_(i) representsthe displacement of the same, and K_(fi) represents the stiffness force.FIG. 9 shows the relationship between <K_(f)>, <K(<U>)>, and <U>.Further, <U^(k)> represents a stack of u_(i) ^(k) corresponding to therespective elements and is written as

u ^(k)=(u ₁ ^(k) , u ₂ ^(k) , . . . , u _(N) ^(k))  (5)

where N represents the number of finite elements.

To calculate K_(fi) ^(k) using u_(i) ^(k) in the above process (6),K_(fi) is denoted as K_(e), and the following equation (6) is solved.

Ke=∫B(x)^(T) DB(x)dx ₁ dx ₂ dx ₃  (6)

Here, <B(<x>)>is a shape matrix that relates the strain tensor to thedisplacement (displacement from nodal point 1 to nodal point n) by theequation (strain tensor)=(shape matrix)·(displacement), and <D> is aproperty matrix that relates the stress tensor to the strain tensor bythe equation (stress tensor)=(property matrix)·(strain tensor).

Further, each u_(m) in u^(k)=(u₁ ^(k), u₂ ^(k), . . . , u_(N) ^(k)) isdescribed as u_(m)=((x1, x2, x3)_(p(m)), (x1, x2, x3)_(q(m)), (x1, x2,x3)_(r(m)), (x1, x2, x3)_(s(m)), [m: 1, 2, . . . , N]. Here, mrepresents the element number, and indicates that the total number ofelements is N. On the other hand, p(m), q(m), r(m), and s(m) indicatethe numbers of the four nodal points forming the element m as atetrahedron. Further, (x1, x2, x3)_(p(m)) represents the displacement atthe nodal point p(m).

By processing the above equation (4), the simulated motion computingunit 507 reconfigures the stiffness matrix <K> according to the shapeand updates it on a frame-by-frame basis on the simulated motioncomputing unit 507 (P402).

In the fourth embodiment, since the stiffness matrix K is generatedusing the spring constants of the imaginary springs, the matrix elementsare simple in configuration. Accordingly, the simulated motion computingunit 507 can perform the computation of equation (4) using the stiffnessmatrix K at high speed. While the computation using the prior artstiffness matrix has had the problem that the computation may diverge,in the fourth embodiment the computation of equation (4) does notdiverge but converges because the stiffness matrix K is formed frommatrix elements of simple configuration.

During the surgical simulation, the image generating unit 503 generatessimulated images of the simulated organ as seen from the simulatedendoscope, based on the position of the simulated organ computed by thesimulated motion computing unit 507. In this way, the image generatingunit 503 generates motion simulated images of the simulated organ andthe simulated surgical instructions 203 including the simulated forceps,as if the images were being actually captured by the simulated endoscope(P403).

The image generating unit 503 displays the simulated images on the imagedisplay device 504, and the surgical simulation operator performs thesurgical simulation while viewing the displayed images (P404).

According to the surgical simulation method and surgical simulator ofthe fourth embodiment described above, the position achieved by themotion of the simulated organ can be computed in real time. Further,since the simulation accuracy of simulated organ deformation can beimproved with the spring forces acting on the simulated organ, asurgical simulation having a high training effect can be achieved.

In the earlier described third embodiment, the stiffness matrix Kdescribes the physical values of the simulated organ and is generatedusing the spring constants of the imaginary inter-membrane springs andin-plane springs. Then, the motion of the simulated membranes iscomputed in real time by using the stiffness matrix K. In the thirdembodiment, the simulated organ may be constructed by using imaginarysprings similar to those used to construct the simulated organ in thefourth embodiment.

DESCRIPTION OF THE REFERENCE NUMERALS

501: MEDICAL IMAGE DATA STORAGE UNIT,

502: VOLUME DATA CONSTRUCTING UNIT,

503: IMAGE GENERATING UNIT,

504: IMAGE DISPLAY DEVICE

505: SURGICAL SIMULATION MODEL DATA UNIT,

506: FORCE SENSING DEVICE,

507: SIMULATED MOTION COMPUTING UNIT,

508: SIMULATED SURGICAL INSTRUMENT,

509: SIMULATED ENDOSCOPE,

510: SIMULATED FORCEPS,

601: ORGAN, 602 a, 602 b, 602 c, . . . : SURFACE NODAL POINTS,

603 a, 603 b, 603 c, . . . : IMAGINARY LINES,

604 a, 604 b, 604 c, . . . : SIMULATED MEMBRANES,

605 a, 605 b, 605 c, . . . : MEMBRANE NODAL POINTS,

606 a, 606 b, 606 c, . . . : IMAGINARY INTER-MEMBRANE SPRINGS,

607 a, 607 b, 607 c, IN-PLANE SPRINGS.

1-7. (canceled)
 8. A surgical simulation method comprising: a forcesensing simulation process in which a computing unit causes a forcesensing device to produce reaction of a simulated organ that matches theposition of a simulated surgical instrument being manipulated by asurgical simulation operator and the position where said simulatedsurgical instrument touches said simulated organ; a simulated motioncomputing process in which said computing unit acquires, from a storageunit, surgical simulation model data for a simulated organ generated byarranging an imaginary spring so as to connect between each nodal pointformed by meshing an organ represented by volume data, computes thereaction of said simulated organ due to a movement of said simulatedsurgical instrument and the touching of said simulated organ with saidsimulated surgical instrument in said force sensing simulation process,and supplies said computed reaction to said force sensing simulationprocess, while at the same time, computing the position achieved by themotion of said simulated organ; an image generation process in whichsaid computing unit generates, based on the position of said simulatedorgan computed in said simulated motion computing process, a simulatedimage of said simulated organ as seen from a simulated endoscope; and animage display process in which said computing unit displays said imagegenerated in said image generation process on a display unit.
 9. Asurgical simulation method as claimed in claim 8, wherein in saidsimulated motion computing process, said computing unit varies aphysical value of said simulated organ according to a deformation causedon said simulated organ by a force applied to said simulated organ, andsaid computing unit computes the reaction of said simulated organ basedon said varied physical value.
 10. A surgical simulation method asclaimed in claim 9, wherein in said simulated motion computing process,said computing unit computes the reaction, f, of said simulated organ byusing the equationf =MÜ+C{dot over (U)}+K(U) where displacement vector U represents thepositional displacement of said simulated organ, stiffness matrix Krepresents the physical value of said simulated organ and is generatedusing a spring constant of said imaginary spring, matrix M is a massmatrix, and matrix C is a viscosity resistance matrix. 11-12. (canceled)13. A surgical simulator comprising: a surgical simulation model dataunit which stores surgical simulation model data for a simulated organgenerated by arranging an imaginary spring so as to connect between eachnodal point formed by meshing an organ represented by volume data; aforce sensing device which produces the reaction of said simulated organthat matches the position of a simulated surgical instrument beingmanipulated by a surgical simulation operator and the position wheresaid simulated surgical instrument touches said simulated organ; asimulated motion computing unit which acquires said surgical simulationmodel data from said surgical simulation model data unit, computes thereaction of said simulated organ due to a movement of said simulatedsurgical instrument and the touching of said simulated organ with saidsimulated surgical instrument at said force sensing device, and suppliessaid computed reaction to said force sensing device, while at the sametime, computing the position achieved by the motion of said simulatedorgan; an image generating unit which generates, based on the positionof said simulated organ computed by said simulated motion computingunit, a simulated image of said simulated organ as seen from a simulatedendoscope; and an image display unit which displays said image generatedby said image generating unit.
 14. A surgical simulator as claimed inclaim 13, wherein said simulated motion computing unit varies a physicalvalue of said simulated organ according to a deformation caused on saidsimulated organ by a force applied to said simulated organ, and computesthe reaction of said simulated organ based on said varied physicalvalue.
 15. A surgical simulator as claimed in claim 14, wherein saidsimulated motion computing unit computes the reaction, f, of saidsimulated organ by using the equationf =MÜ+C{dot over (U)}+K(U) where displacement vector U represents thepositional displacement of said simulated organ, stiffness matrix Krepresents the physical value of said simulated organ and is generatedusing a spring constant of said imaginary spring, matrix M is a massmatrix, and matrix C is a viscosity resistance matrix.